Portable device

ABSTRACT

A portable computing device is disclosed which may include a base element, a lid element, a first motion sensor, a second motion sensor, a processor, and an eye tracking system. The first motion sensor may be disposed in the base element. The second motion sensor may be disposed in the lid element. The processor may be configured to control the first motion sensor to detect first motion information, control the second motion sensor to detect second motion information; and determine final motion information based at least in part on the first motion information and the second motion information. The eye tracking system may be configured to determine a gaze position of a user based at least in part on the final motion information, wherein the processor is further configured to execute one or more control processes based at least in part on the determined gaze position meeting a predetermined condition.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 15/259,538, filed Sep. 8, 2016, which is a continuation of U.S.Pat. No. 9,465,415, filed May 2, 2014, which is a national stage entryof PCT/EP2012/069445, filed Feb. 10, 2012, which claims priority to U.S.Prov. Pat. App. No. 61/556,505, filed Nov. 7, 2011, and European Pat.App. No. 11187816.1, filed Nov. 4, 2011.

BACKGROUND OF THE INVENTION

Embodiments of the present invention relate generally to automatictracking of distinctive features of users operating electronicequipment.

Laptops with integrated eye-trackers are known. Unfortunately, existingsolutions are comparatively bulky, and therefore the laptop designbecomes relatively thick, i.e., in closed/inactive mode, the laptop hasa rather high profile. Naturally, this is undesired because portabledevices, such as laptops, in general should be as compact and slim aspossible. Nevertheless, this bulkiness can sometimes be an advantage.Because the optical remote sensing systems of today's eye-trackers attimes consume relatively large amounts of power, substantial thermalpower dissipation must be available. A larger volume is more capable ofdissipating this heat than a smaller volume. Therefore, the trend ofeach generation of laptop becoming thinner than the previous generationis problematic because the available space for thermal dissipation inthe lid of the laptop becomes more and more limited. This places severeconstraints on which components that can be used to implementeye-trackers and similar devices.

U.S. Pat. Pub. No. 2005/0110887 shows an example of a mobilecommunication terminal with a main body and a foldable display body. Acamera is positioned in a hinge unit which connects the display body tothe main body. The camera is rotatable around the hinge axis to registerimages of the user as well as subjects/objects in front of the user.Thus, versatile camera function is attained. However, since the camerais freely rotatable relative to both the main body and the display bodyit may be complicated to use the camera for many purposes, such as foreye-tracking.

In the light of the above, embodiments of the invention provide acompact and yet thermally and power-efficient portable device that isequipped with an optical remote sensing system for eye/gaze tracking,gesture detection, facial feature tracking, and/or user identificationthrough face or iris recognition or hand gesture detection.

BRIEF SUMMARY OF THE INVENTION

One object of various embodiments herein is to mitigate the aboveproblems and provide a slim portable device with an integrated opticalremote sensing system for eye/gaze tracking and/or gesture tracking.

In some embodiments, the above object is achieved by a first laptop orother device, wherein the first laptop or other device has a first partwhich includes a recess. The recess is arranged relative to a positionof the optical remote sensing system such that, when the laptop lid isin the closed position, the optical remote sensing system is at leastpartly disposed in the recess.

This portable device is advantageous because it enables an overall slimdesign to be combined with an adequate cooling volume for the opticalremote sensing system. This is a desirable feature in any portabledevice, including: laptops, note books, ultrabooks, tablets withkeyboards, personal digital assistants, and smartphones.

According to one embodiment, the recess is co-located with a first pieceof the hinge means. The optical remote sensing system is furtherdisposed in a projection of the second part where a second piece of thehinge means is located. The first and second pieces of the hinge meansrepresent a pivot axis via which the first and second parts of theportable device are interconnected. Such an integration of the opticalremote sensing system in the hinge means is desirable because itprovides a volume sufficient to ensure adequate cooling of the opticalremote sensing system while the tracking functionality can be integratedin an essentially indiscernible manner into the portable device.

According to another embodiment, the first and second parts of theportable device are interconnected via the hinge means along a proximalside of the second part. The optical remote sensing system is heredisposed in a projection extending along a distal side of the secondpart, where the distal side is opposite to the proximal side. In alaptop implementation, this means that the optical remote sensing systemis located above the screen when the device is arranged in theopen/active mode. Such a positioning is especially advantageous if auser's gestures are to be interpreted by the optical remote sensingsystem.

According to yet another embodiment, the second essentially flat surfaceof the second part includes a display unit (e.g., an LCD screen)configured to present information to the user. It is further preferableif the first essentially flat surface of the first part includes akeyboard configured to receive input commands from the user. Hence, theoptical remote sensing system is included in the same part as thedisplay unit.

According to still another embodiment, the optical remote sensing systemis arranged such that a view angle thereof has a fixed spatial relationto the display unit irrespective of an orientation of the second partrelative to the first part. Of course, this is desirable because therebyit is fairly straightforward to determine the user's point of regard onthe display unit based on data registered by the optical remote sensingsystem.

According to a further embodiment, the first part is a base element andthe second part is a lid element. During operation, the base element isconfigured to be placed on an essentially flat sup-porting surface(e.g., a desk) while the lid element is positioned upright, so that itsessentially flat inner surface (typically containing a display unit) isvisible to the user.

According to other embodiments, the optical remote sensing systemincludes an image registering unit (e.g., a still camera or a videocamera), and preferably, at least one illuminator configured toilluminate the user. A combined camera-and-illuminator is generallyadvantageous for cost efficiency. In eye-tracker implementations it isalso desirable that one or more light sources be arranged close to theoptical axis of the image registering unit. Embodiments of theinvention, however, are likewise applicable to designs where the lightsource and the image registering unit are separated from one another. Inany case, it is generally preferable that the optical remote sensingsystem includes an optical filter, which is arranged in front of anilluminator and/or an image registering unit therein, and which opticalfilter is configured to block visible light however is transparent tonear-infrared (NIR) light. Namely, as will be discussed below, NIR lightis desirable, whereas visible light may disturb the user.

According to still another embodiment, at least one of the at least oneilluminator is configured to produce structured light, which whenregistered by the image registering unit, causes resulting data to becreated, which resulting data are adapted for generating a depth map ofthe user. This is advantageous both when interpreting gestures and ineye-tracking, for instance when selecting a relevant image segment toprocess.

According to yet another embodiment, at least one of the at least oneilluminator is configured to produce near-infrared light. Namely, thistype of light is relatively uncomplicated to detect by a camera, howeverinvisible to the human eye.

In some embodiments, at least one of the at least one illuminator isconfigured to produce a light beam whose direction is controllable, sothat a varying position of the user can be tracked. Directional opticalilluminators are advantageous relative to static ditto because, at eachpoint in time, the directional illuminator only illuminates a fractionof a surface inside a volume within which the subject moves. Thereby,power is conserved corresponding to the size of the non-illuminatedsurface that would otherwise have been illuminated.

According to another embodiment, at least one of the at least oneilluminator is based on LED (Light Emitting Diode) technology. Namely,LEDs represent energy-efficient, compact and reliable light sources.

According to still another embodiment, at least one of the at least oneilluminator is configured to produce coherent light. Coherent lightsources (e.g., lasers) are desirable, since such a light source may becombined with diffractive optical elements to transform a light beaminto a desired spatial pattern. Thus, the illumination can be controlledvery efficiently, for instance to follow a position of the user.

According to a further embodiment, the optical remote sensing systemincludes an eye tracker configured to repeatedly determine a position ofat least one eye of the user and/or repeatedly determine a point ofregard of the user relative to the portable device. Thereby, it ispossible to generate input commands to a laptop based on the user'socular activity.

Thus, in one embodiment, a portable computing device is provided. Theportable computing device may include a base element, a lid element, afirst motion sensor disposed in the base element, a second motion sensordisposed in the lid element, a processor, and an eye tracking system.The processor may be configured to control the first motion sensor todetect first motion information, control the second motion sensor todetect second motion information, and determine final motion informationbased at least in part on the first motion information and the secondmotion information. The eye tracking system may be configured todetermine a gaze position of a user based at least in part on the finalmotion information. The processor may be further configured to executeone or more control processes based at least in part on the determinedgaze position meeting a predetermined condition.

In another embodiment, a method for controlling an eye tracking systemof a portable device is provided. The method may include detecting firstmotion information with at least a first motion sensor disposed in afirst part of a portable device. The method may also include detectingsecond motion information with at least a second motion sensor disposedin a second part of the portable device. The method may further includedetermining final motion information based on the first motioninformation and the second motion information. The method may moreoverinclude determining a gaze position of a user based on at least in partof the final motion information. The method may furthermore includeexecuting one or more control processes based at least in part on thedetermined gaze position meeting a predetermined condition.

In another embodiment, a non-transitory computer readable medium havingstored thereon a program for controlling an eye tracking system of aportable device is provided. The program may include a step of detectingfirst motion information with at least a first motion sensor disposed ina first part of a portable device. The program may also include a stepof detecting second motion information with at least a second motionsensor disposed in a second part of the portable device. The program mayfurther include a step of determining final motion information based onthe first motion information and the second motion information. Theprogram may additionally include determining a gaze position of a userbased on at least in part of the final motion information. The programmay moreover include executing one or more control processes based atleast in part on the determined gaze position meeting a predeterminedcondition.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention are described in conjunction withthe appended figures:

FIG. 1A and FIG. 1B show side views of a portable device according to afirst embodiment of the invention;

FIG. 2 shows a top view of a first part of the portable device in FIGS.1A and 1B;

FIG. 3A, and FIG. 3B show side views of a portable device according to asecond embodiment of the invention;

FIG. 4 shows a top view of a first part of the portable device in FIGS.3A and 3B;

FIG. 5 illustrates, in further detail, the first embodiment of theinvention depicted in FIGS. 1A, 1B, and 2;

FIG. 6 illustrates, in further detail, the second embodiment of theinvention depicted in FIGS. 3A, 3B, and 4;

FIG. 7 illustrates a side view of a portable device including at leasttwo motion sensors according to a third embodiment of the invention;

FIG. 8 illustrates a hardware configuration of the portable deviceaccording to a third embodiment of the invention;

FIG. 9 illustrates a front view of the display unit of the portabledevices according to a third embodiment of the invention; and

FIG. 10 illustrates a flow chart describing a processing procedureaccording to a third embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

The ensuing description provides exemplary embodiments only, and is notintended to limit the scope, applicability or configuration of thedisclosure. Rather, the ensuing description of the exemplary embodimentswill provide those skilled in the art with an enabling description forimplementing one or more exemplary embodiments. It being understood thatvarious changes may be made in the function and arrangement of elementswithout departing from the spirit and scope of the invention as setforth in the appended claims.

For example, any detail discussed with regard to one embodiment may ormay not be present in all contemplated versions of that embodiment.Likewise, any detail discussed with regard to one embodiment may or maynot be present in all contemplated versions of other embodimentsdiscussed herein. Finally, the absence of discussion of any detail withregard to embodiment herein shall be an implicit recognition that suchdetail may or may not be present in any version of any embodimentdiscussed herein.

Specific details are given in the following description to provide athorough understanding of the embodiments. However, it will beunderstood by one of ordinary skill in the art that the embodiments maybe practiced without these specific details. For example, circuits,systems, networks, processes, and other elements in the invention may beshown as components in block diagram form in order not to obscure theembodiments in unnecessary detail. In other instances, well-knowncircuits, processes, algorithms, structures, and techniques may be shownwithout unnecessary detail in order to avoid obscuring the embodiments.

Also, it is noted that individual embodiments may be described as aprocess which is depicted as a flowchart, a flow diagram, a data flowdiagram, a structure diagram, or a block diagram. Although a flowchartmay describe the operations as a sequential process, many of theoperations can be performed in parallel or concurrently. In addition,the order of the operations may be re-arranged. A process may beterminated when its operations are completed, but could have additionalsteps not discussed or included in a figure. Furthermore, not alloperations in any particularly described process may occur in allembodiments. A process may correspond to a method, a function, aprocedure, a subroutine, a subprogram, etc. When a process correspondsto a function, its termination corresponds to a return of the functionto the calling function or the main function.

The term “computer readable medium” or “machine readable medium”includes, but is not limited to transitory and non-transitory, portableor fixed storage devices, optical storage devices, wireless channels andvarious other mediums capable of storing, containing or carryinginstruction(s) and/or data. A code segment or machine-executableinstructions may represent a procedure, a function, a subprogram, aprogram, a routine, a subroutine, a module, a software package, a class,or any combination of instructions, data structures, or programstatements. A code segment may be coupled to another code segment or ahardware circuit by passing and/or receiving information, data,arguments, parameters, or memory contents. Information, arguments,parameters, data, etc. may be passed, forwarded, or transmitted via anysuitable means including memory sharing, message passing, token passing,network transmission, etc.

Furthermore, embodiments of the invention may be implemented, at leastin part, either manually or automatically. Manual or automaticimplementations may be executed, or at least assisted, through the useof machines, hardware, software, firmware, middleware, microcode,hardware description languages, or any combination thereof. Whenimplemented in software, firmware, middleware or microcode, the programcode or code segments to perform the necessary tasks may be stored in acomputer or machine readable medium. A processor(s) may perform thenecessary tasks. The terms “comprises,” “comprising,” “includes,”“including,” and other terms herein specify the presence of statedfeatures, integers, steps, or components. However, these terms do notpreclude the presence or addition of one or more additional features,integers, steps, and/or components or groups thereof.

We refer initially to FIGS. 1A and 1B, which show side views of aportable device including a remote optical sensing system 300 accordingto a first embodiment of the invention. For convenience, the portabledevice is here embodied as a laptop. However, according to variousembodiments of the invention, the portable device may equally well berepresented by an alternative electronic device, such as a note book, anultrabook, a tablet with a keyboard, a personal digital assistant (PDA)or a smartphone.

The proposed portable device includes a first part 110 (here representedby a laptop base element) and a second part 120 (here represented by alaptop lid element). The second part 120, in turn, includes the opticalremote sensing system 300. As is common in laptops, the second part 120is pivotably attached to the first part 110 via a hinge means 115, suchthat the portable device may be arranged in an open and a closedposition respectively. FIG. 1A illustrates the closed position(predominantly used when the device is inactive) and FIG. 1B illustratesthe open position (the typical position when the device is activated).The optical remote sensing system 300 is configured to track at leastone distinctive feature of a user of the portable device when the deviceis active and arranged in the open position. The at least onedistinctive feature may be an eye, a facial feature and/or a limb of theuser, such as his/her hand. Thereby, the optical remote sensing system300 may be used for eye-, gaze, gesture and/or facial feature trackingand/or user identification through face or iris recognition or handgesture detection.

The first and second parts 110 and 120 have a respective essentiallyflat inner surface 111 and 121. When the portable device is arranged inthe closed position, the essentially flat inner surfaces 111 and 121 areparallel and face one another, as can be seen in FIG. 1A. The innersurfaces 111 and 121 are not entirely flat. Namely, for example, thefirst part 110 has a recess 112 a in the essentially flat inner surface111, which recess 112 a is arranged relative to a position of theoptical remote sensing system 300 in the second part 120, such that, inthe closed position, the first part 110 receives a projection 125 a ofthe second part 120, which projection 125 a includes the optical remotesensing system 300. As a result, the optical remote sensing system 300is at least partly contained in the recess 112 a when the portabledevice is arranged in the closed position (i.e., normally the inactivemode).

The first essentially flat surface 111 of the first part 110 preferablyalso includes a keyboard configured to receive input commands from theuser. Moreover, the second essentially flat surface 121 of the secondpart 120 preferably includes a display unit 122 (see FIG. 5) configuredto present information to the user, such as text, graphics, imagesand/or video.

FIG. 2 shows a top view of the first part 110 of the portable device inFIGS. 1A and 1B. The recess 112 a preferably extends between a pair ofhinge means 115 (symbolically illustrated with dashed lines), which inturn are arranged in proximity to the portable device's sides. Above thehinge means 115, the second part 120 preferably extends to a full widthof the first part 110 (not shown). As is apparent from FIGS. 1A, 1B and2, the recess 112 a is co-located with a first piece of the hinge means115, say a female piece. A second piece of the hinge means 115, say amale piece, is located on the projection 125 a of the second part 120.The first and second pieces of the hinge means 115 represent a pivotaxis via which the first and second parts 110 and 120 areinterconnected. Hence, as is shown in FIG. 1A (and in further detail inFIG. 5), when the portable device is closed, the projection 125 a isconfigured to swing down into the recess 112 a, so that the recess 112 ahouses the optical remote sensing system 300, at least partly. It istechnically possible that the recess 112 a is represented by a complete,or partial, cut-out in the first part 110. The dashed line 113 in FIG. 2shows an example of where an edge of such a partial cut-out may belocated. Nevertheless, it is desirable that a bottom part of the recess112 a covers at least any optical filter in front of an illuminatorand/or an image registering unit in the optical remote sensing system300 when the portable device is closed. For optimal compactness of theportable device, the recess 112 a should be as deep as possible; i.e.,in closed mode, as much as possible of the optical remote sensing system300 should be contained in the recess 112 a (or 112 b, see below). Thismeans that the bottom of the recess 112 a may be a 2 mm thick tonguecovering a front-most side of the optical remote sensing system 300 whenthe portable device is closed.

FIGS. 3A and 3B show side views of a portable device including anoptical remote sensing system 300 according to a second embodiment ofthe invention. FIG. 4 shows a top view of the first part 110 of theportable device in FIGS. 3A and 3B. In FIGS. 3A, 3B and 4 all elementshaving labels which also occur in FIGS. 1A, 1B and/or 2 represent thesame entities as those described above with reference to FIGS. 1A, 1Band/or 2.

Again, the portable device has first and second parts 110 and 120 thatare pivotably attached to one another, such that the portable device maybe arranged in an open and a closed position respectively. In this case,however, the optical remote sensing system 300 is not co-located withthe hinge means 115. Instead, the optical remote sensing system 300 isdisposed in a projection 125 b extending along a distal side of thesecond part 120, whereas the hinge means 115 are arranged along aproximal side of the second part 120, which proximal and distal sidesare opposite to one another.

FIG. 5 illustrates in further detail the first embodiment of theinvention depicted in FIGS. 1A, 1B and 2. Here, the optical remotesensing system 300 is arranged in the projection 125 a of the secondpart 120, which is co-located with the hinge means 115. Preferably, theoptical remote sensing system 300 is further arranged such that a viewangle α thereof has a fixed spatial relation to the display unit 122irrespective of an orientation of the second part 120 relative to thefirst part 110. The ideal view angle α depends on the specificcharacteristics of an image registering unit included in the opticalremote sensing system 300. However, preferably a has a value in therange 50° to 65°, more preferably 56° to 60°, and most preferably a 58°.Namely, thereby the risk that the user's hands block the optical remotesensing system 300 is relatively low, and at the same time, anacceptable angle towards the user's face is attained for the mostcommonly used inclinations of the second part 120 relative to the firstpart 110 in the active mode. Hence, for example the user's point ofregard on the display unit 122 can be determined repeatedly based ondata from an eye-tracker of the optical remote sensing system 300.

FIG. 6 illustrates in further detail the second embodiment of theinvention depicted in FIGS. 3A, 3B and 4. Here, the optical remotesensing system 300 is arranged in the projection 125 b of the secondpart 120, which extends along a side of the second part that is oppositeto the side where the hinge means 115 are located. Also in thisembodiment, the optical remote sensing system 300 is preferably arrangedsuch that a view angle α thereof has a fixed spatial relation to thedisplay unit 122 irrespective of an orientation of the second part 120relative to the first part 110. Nevertheless, in this case, the viewangle α preferably has a value in the range 80° to 100°, and mostpreferably α=90°. Thus, the optical remote sensing system 300 may beefficiently used for eye- and/or gaze tracking as well as forinterpretation of facial expressions and/or gestures.

Irrespective of whether the optical remote sensing system 300 isco-located with the hinge means 115 (as in FIG. 5), or arranged inproximity to a distal side thereof (as in FIG. 6), it is generallydesirable that an illuminator and/or an image registering unit of theoptical remote sensing system 300 is covered by an optical filter,which, preferably, is configured to block visible light while permittingNIR light to pass through.

Additionally, regardless of the location of the optical remote sensingsystem 300, according to some embodiments of the invention, the opticalremote sensing system 300 includes an image registering unit and atleast one illuminator configured to illuminate the user. The imageregistering unit, in turn, may contain a still and/or a video cameraconfigured to capture image data representing the user of the portabledevice, such as images of his/her eyes.

It is further advantageous if at least one of the at least oneilluminator is configured to produce structured light, which whenreflected against the user and registered by the image registering unitcauses resulting data to be created, which resulting data are adaptedfor generating a depth map of the user. Depth maps are advantageous bothwhen interpreting gestures and during eye-tracking, for instance whenselecting a relevant image segment to process.

Moreover, one or more of the at least one illuminator may be configuredto produce (NIR) light. NIR light is advantageous because it isrelatively uncomplicated to detect by a camera and because it isinvisible to the human eye. Thus, NIR light does not disturb the user.

It is further advantageous if one or more of the at least oneilluminator is configured to produce a light beam whose direction iscontrollable to track a varying position of the user. If at least one ofthe at least one illuminator is configured to produce coherent light,diffractive optical elements (DOE) may be used to transform the lightbeam into a desired spatial pattern. Thus, the illumination can becontrolled very efficiently, for instance to follow a position of theuser.

Alternatively, or as a complement, at least one of the at least oneilluminator may be based on LED technology. LEDs are desirable lightsources since they are energy-efficient, compact and reliable.

In a third embodiment, a portable device having two portions connectedvia a hinge, such as laptop, is capable of opening to a large angle(e.g., over 120 degrees) with respect to the surface of base element(shown as the first part 110). Some convertible laptops are evenequipped with a rotatable display, such that when the laptop is in theopen position, the display portion can be rotated, twisted and tilted.Where the portable device is equipped with an eye tracker, the eyetracker is normally mounted towards the base of the display portion, anda large open angle, or extremely tilted, twisted, or rotated displaypresents a problem in that the eye tracker (shown as optical remotesensing system 300) may not be able to achieve optimal eye trackingperformance.

For the purpose of this document, the term “open angle” is intended torefer to the angle representing the orientation of the first part 110 tothe second part 120. In other words, the degree of openness of a laptopor similar portable device.

The performance of the eye tracker, such as precision or accuracy of eyetracking, may be severely affected as the gaze positions may be tooclose to the edge of the second part 120 or even out of the gazetracking area on the display unit 122 when, for example, the open angleis too large. Therefore, there is a need to determine the open angle ofthe portable device and ultimately determine the orientation of the lidelement (shown as the second part 120) and dynamically control and/orrecalibrate the eye tracker based on this and/or any other system eventsassociated with the performance of the eye tracker. The exact positionand/or orientation of the second part 120 in relation to the opticalremote sensing system 300 may be stored in any form of computer readablestorage of the portable device. This may allow for improved powerefficiency and more accurate and precise eye tracking performance.

FIG. 7 show views of a portable device including at least two motionsensors 401, 402 according to a third embodiment of the invention. Inthe third embodiment, the portable device includes two or more motionsensors equipped in the second part 120 (for example a laptop lidelement) and the first part 110 (for example a laptop base element)respectively. The number of motion sensors may be more than two, withdifferent configurations according to their placement accordingly.

For this non-limiting example, the motion sensor may be any kind ofInertial Measurement Unit (IMU) or Microelectromechanical (MEMS) system,such as an accelerometer, gyroscope, and/or magnetometer. The motionsensor can also be an inertial measurement module coupled with aplurality of the aforementioned motion sensors or integrated as aSystem-in-Package (SiP).

The accelerometer may be an electromechanical device that measuresacceleration forces, as would be readily understood by a person of skillin the art. These forces may be static, like the constant force ofgravity in a case where the accelerometer is not moved or vibrated for aperiod of time, or they could be dynamic—caused by moving or vibratingthe accelerometer. The accelerometer may be of different types, such asa digital accelerometer or analog accelerometer. The specifications ofthe accelerometers, such as number of measurement axes, output range,sensitivity and dynamic range, can be manually set by a user orautomatically set according to the usage of the portable device.

The gyroscope (or gyro sensor) senses angular velocity from the Coriolisforce applied to a vibrating object. And the vibrating object may be thesecond part 120, the first part 110, or may be the portable device ingeneral. The types of gyroscopes are also not limited, and may includeTuning Fork Gyroscopes, Vibrating-Wheel Gyroscopes, Wine Glass ResonatorGyroscopes or Foucault Pendulum Gyroscopes. The gyroscopes may be astand-alone chip module communicatively coupled to the system bus of thecircuitry of the portable device or may be printed onto the circuitboard (e.g., motherboard) of the portable device using photolithography.Again, the specification of the gyroscopes, such as measurement range,number of sensing axes, linearity or nonlinearity, shock survivability,angular random walk (ARW), bias, bias drift and bias instability, can bemanually set by a user or automatically set according to the usage ofthe portable device.

Alternatively, the motion sensor of 401 and/or 402 may be a compositemodule coupled with one or more accelerometers and/or one or moregyroscopes. Then the measurement of acceleration force(s) and angularvelocity is possible.

The one or more motion sensors in the second part 120 may be preferablyplaced at the bottom or substantially close to the bottom of the secondpart 120, where the bottom is the edge of the second part 120 mostclosely located to the first part 110, which means the one or moremotion sensors may be substantially close to the hinge means 115.“Substantially close” in this context means proximate to the hinge anddistal to the opposite edge of the second part 120. In some embodiments,this may mean the one or more motion sensors may be located below thescreen of the second part 120. It is advantageous to have such placementto minimize risk of low determination accuracy caused by accidental orunwanted vibration of the second part 120, which may affect theprecision of angle determination (details will be described in thefollowing paragraphs). The second part 120 of a portable device (e.g.,laptop) may be of such limited thickness that minor vibrations may causeshaking of second part 120. However, the placement of the one or moremotion sensors is not limited to the aforementioned position, it can beany place in second part 120.

The motion sensor may be integrated into the motherboard of the portabledevice or circuitry of the first part 110 or second part 120.Alternatively, the motion sensor may also be placed externally withrespect to the enclosure (e.g., either second part 120 or first part110) of the portable device. In such circumstance, the motion sensor maybe equipped as a module of a Raspberry Pi® that communicatively coupledto the laptop via any I/O interface (e.g., Universal Serial Bus (USB))or preferably has wireless connectivity (e.g., Bluetooth™, WiFi) fordata transmission. Other similar forms of embedded devices are possible.

FIG. 8 is a block diagram 800 illustrating a hardware configurationincluding therein the portable device in accordance with an embodimentof the present disclosure. Referring to FIG. 8, the portable device mayinclude, but not limited to, a bus 810, at least one processor 801, atleast one input device 802, at least one output device 803, at least onestorage device 804, at least one computer readable storage media reader805, an eye tracker 806, a communication system 807 equipped with atleast Bluetooth and/or WiFi connectivity, at least two motion sensors808 (either accelerometer(s) and/or gyroscope(s)) and working memory 809storing operating system and other codes or programs.

In a non-limiting example, as shown in FIG. 7, under the control of theprocessor, the motion sensor 401 in the first part 110 acquires a firstangle value (V_(base)) and the motion sensor 402 in the second part 120acquires a second angle value (V_(lid)). The acquisition of the anglevalues for either first part 110 and the lid element may be in the formof asynchronization or synchronization in real time, or in a periodicform for designated or predetermined time intervals. The first anglevalue (V_(base)) and the second angle value (V_(lid)) is transmitted tothe processor. Then the open angle value (V_(close up)), between thesurface of the first part 110 and the surface of the second part 120, iscalculated in the processor by using a first angle value (V_(base)) andthe second angle value (Vlid). After the calculation of the open anglevalue (V_(close up)), the 3D coordinates value (position in the 3Dspace) of at least three points on the surface of the second part 120,with respect to a predetermined position at the bottom of the displayunit 122 that substantially closed to the position of the camera of theeye tracker, are calculated by the processor (based at least in part onthe data from the motion sensors). Alternatively, the predeterminedposition can be specified by the user. The calculation of the 3Dcoordinates value is based on the acquired the first angle value(V_(base)), the second angle value (V_(lid)), the open angle value(V_(close up)) and the pre-known dimension information of the displayunit 122. Alternatively, the dimension information may be specified bythe user. As shown in the FIG. 9, It is advantageous that aforementionedthree or more points should include at least two points thatsubstantially close to the top right corner (shown as C in FIG. 9) ofthe display unit 122 and the top left corner (shown as A in FIG. 9) ofthe display unit 122 respectively, and at least one point that issubstantially closed to the lower left corner (shown as B in FIG. 9) orlower right corner of the display unit 122. “Substantially close” inthis context may mean proximate to such positions, and/or the mostextreme corners of the visible-to-the-user portion of the display unit.Preferably, the distance between each of those three points should be aslarge as possible.

Then a final motion information is determined in the processor viaarithmetic calculation by using the V_(close up) and the 3D coordinates'values of the at least three points. Optionally, the final motioninformation may be determined by only using either of the V_(close up)or the 3D coordinates' values of the at least three points. The finalmotion information is sent to the eye tracker. Alternatively, the finalmotion information may be determined before ahead if it is within apredetermined threshold value. The predetermined threshold value mayindicate a range of open angle value. In an extreme circumstance thatthe open angle of the laptop or the twisting angle of the second part120 may be too large to be used for the use of gaze determination (willbe described in the following). After the acquisition of the finalmotion information, eye tracker is controlled to determine the user'sgaze positions of at least one eye relative to the display unit 122.Besides, the eye tracker may also alternatively determine the head poseof the user with respect to the display unit 122 and take the head poseinformation into account of determination of gaze positions. Here, headpose may be determined based on one or more images captured by theoptical remote sensing. The head pose may be defined by the position andthe orientation of the head in three-dimensional space at someparticular time. The head pose may be determined by examining one ormore facial features (for instance mouth, nose, etc.) and theirpositions and orientations relative to one another. In non-limitingdetermination condition, under the control of the processor, the eyetracker is controlled to (a) determine if the gaze positions cannot bedetermined at all with respect to the display unit 122; or (b) determineif the gaze positions cannot be determined in a predetermined area withrespect to the display unit 122 (e.g., corner of the display unit, orarea that is close to the edge of the display unit 122). Note that “gazeposition” is used herein to not only describe actual determined gazepositions, but also may include a description of a scenario where thegaze position cannot be determined with respect to the display unit 122or otherwise.

If the result of the aforementioned determination is positive, then theeye tracker is controlled for the following one or both of theexecutions: (i the eye tracker may be power ON or OFF or pause the eyetracker for image collection or adjust one or more photographyparameters (e.g., frame rate, exposure, ISO and etc.) for imagecollection or start over the calibration process of the eye tracker;(ii) one or more system events associated with the eye tracker may becontrolled, such as the function of the application on the portabledevice. Other executions are possible, not limited to the aforementionedexecutions.

Next, referring to the flowchart of FIG. 10, a processing procedureaccording to the third embodiment of the invention. At step 1010, theportable device starts the process. At step 1020, the motion sensor inthe first part 110 and the second part 120 are controlled to determineangle value V_(lid), V_(base) respectively. At step 1030, the open anglevalue of the portable device is calculated based on the determined twoangle values V_(lid), V_(base) at step 1020.

At step 1040, the 3D coordinate values of three or more points on thedisplay unit 122 are calculated in the processor. At step 1050, a finalmotion information is determined by using calculated the 3D coordinatevalues and the open angle value. At step 1060, after the determinationof the final information, the final information is further determinedwhether it is within the threshold value. If the final motioninformation is within threshold value, then proceeds to step 1070; ifthe final motion information is not within the threshold value.

At step 1070, the eye tracker is controlled to determine gaze positionsof the user with respect to the display unit 122. At step 1080, the gazepositions are used to (a) determine if the gaze positions cannot bedetermined at all with respect to the display unit 122; or (b) determineif the gaze positions cannot be determined in a predetermined area withrespect to the display unit 122 (e.g., corner of the display unit, orarea that closed to the edge of the display unit 122. And if thedetermination result is position. If the determination result isnegative. At step 1090, the eye tracker is controlled to executecorresponding control process.

Embodiments of the invention has now been described in detail for thepurposes of clarity and understanding. However, it will be appreciatedthat certain changes and modifications may be practiced within the scopeof the appended claims.

What is claimed is:
 1. A portable computing device, comprising: a baseelement; a lid element; a first motion sensor disposed in the baseelement; a second motion sensor disposed in the lid element; a processorconfigured to: control the first motion sensor to detect first motioninformation; control the second motion sensor to detect second motioninformation; and determine final motion information based at least inpart on the first motion information and the second motion information;and an eye tracking system configured to determine a gaze position of auser based at least in part on the final motion information, wherein theprocessor is further configured to execute one or more control processesbased at least in part on the determined gaze position meeting apredetermined condition.
 2. The portable computing device according toclaim 1, wherein: the first motion information comprises a first anglevalue between the gravitational force exerted by the Earth and a planeof the lid element; the second motion information comprises a secondangle value between the gravitational force exerted by the Earth and aplane of the base element; and the final motion information comprises athird angle value derived from at least the first angle value and thesecond angle value.
 3. The portable computing device according to claim1, wherein the first motion sensor and the second motion sensor eachcomprise an accelerometer.
 4. The portable computing device according toclaim 3, wherein the second motion sensor is located below a displayunit of the lid element and proximate to a hinge means rotatablycoupling the base element with the lid element.
 5. The portablecomputing device according to claim 1, wherein the one or more controlprocesses include a selection from a group consisting of: turning offthe eye tracking system; pausing image collection of the eye trackingsystem; adjusting one or more photography parameters of the eye trackingsystem; and controlling one or more system events.
 6. The portablecomputing device according to claim 1, wherein the predeterminedcondition is selected from a group consisting of: the gaze positioncannot be determined; and the gaze position cannot be determined in apredetermined area with respect to a display unit of the lid element. 7.The portable computing device according to claim 6, wherein thepredetermined area is smaller than an entirety of a display area of thedisplay unit.
 8. The portable computing device according to claim 1,wherein the processor is further configured to determine 3D coordinatevalues of three or more positions on a display unit of the lid elementbased at least in part on at least one of the first motion information,the second motion information, or the final motion information.
 9. Theportable computing device according to claim 8, wherein the three ormore positions include: at least one position proximate to a top rightcorner of the display unit; at least one position proximate to a topleft corner of the display unit; and at least one position proximate toa lower left corner or a lower right corner of the display unit.
 10. Theportable computing device according to claim 1, wherein the eye trackingsystem is further configured to determine head pose information of theuser with respect to the display unit.
 11. The portable computing deviceaccording to claim 10, wherein the determination of the gaze position isfurther based on the head pose information.
 12. A method for controllingan eye tracking system of a portable device, wherein the methodcomprises: detecting first motion information with at least a firstmotion sensor disposed in a first part of a portable device; detectingsecond motion information with at least a second motion sensor disposedin a second part of the portable device; determining final motioninformation based on the first motion information and the second motioninformation; determining a gaze position of a user based on at least inpart of the final motion information; and executing one or more controlprocesses based at least in part on the determined gaze position meetinga predetermined condition.
 13. The method of claim 12, wherein: thefirst part comprises a base element of the portable device; and thesecond part comprises a lid element of the portable device.
 14. Themethod of claim 13, wherein: the first motion information comprises afirst angle value between the gravitational force exerted by the Earthand a plane of the base element; the second motion information comprisesa second angle value between the gravitational force exerted by theEarth and a plane of the lid element; and the final motion informationcomprises a third angle value derived from at least the first anglevalue and the second angle value.
 15. The method of claim 12, whereinthe first motion sensor and the second motion sensor each comprise anaccelerometer.
 16. The method of claim 13, wherein the second motionsensor is located below a display unit of the lid element and proximateto a hinge means rotatably coupling the base element with the lidelement.
 17. The method of claim 12, wherein the one or more controlprocesses include a selection from a group consisting of: turning off aneye tracking system; pausing image collection of the eye trackingsystem; adjusting one or more photography parameters of the eye trackingsystem; and controlling one or more system events.
 18. The method ofclaim 13, wherein the predetermined condition is selected from a groupconsisting of: the gaze position cannot be determined; and the gazeposition cannot be determined in a predetermined area with respect to adisplay unit of the lid element.
 19. The method of claim 18, wherein thepredetermined area is smaller than display area of the display unit. 20.The method of claim 13, wherein the method further comprises:determining 3D coordinate values of three or more positions on a displayunit of the lid element based at least in part on at least one of thefirst motion information, the second motion information, or the finalmotion information.
 21. The method of claim 20, wherein the three ormore positions includes at least one is substantially close to the topright corner of the display unit, at least one is substantially close tothe top left corner of the display unit and at least one substantiallyclose to the lower right or left corner of the display unit.
 22. Themethod of claim 13, wherein the method further comprises: determininghead pose information of the user with respect to a display unit of thelid element.
 23. The method of claim 22, wherein the determination ofthe gaze position is further based on the head pose information.
 24. Anon-transitory computer readable medium having stored thereon a programfor controlling an eye tracking system of a portable device comprisingthe steps of: detecting first motion information with at least a firstmotion sensor disposed in a first part of a portable device; detectingsecond motion information with at least a second motion sensor disposedin a second part of the portable device; determining final motioninformation based on the first motion information and the second motioninformation; determining a gaze position of a user based on at least inpart of the final motion information; and executing one or more controlprocesses based at least in part on the determined gaze position meetinga predetermined condition.
 25. A non-transitory computer readable mediumhaving stored thereon a program for controlling an eye tracking systemof a portable device of claim 24, wherein: the first part comprises abase element of the portable device; and the second part comprises a lidelement of the portable device.